We use a mix of Servlet and Webflux spring applications. In Spring Boot 3.0 there are differences in how traces are logged and how they are actually recorded on the tracing backend. The logging differences makes it especially difficult to use tools like Grafana and associate trace IDs from Loki since the position of that trace ID seems to move depending on the app type.

I have created a simple sample to reproduce this issue here: https://github.com/braunsonm/spring-inconsistency-sample. In this sample the two applications are identical except one is servlet and the other is webflux. The sample endpoint uses a RestTemplate and WebClient respectively to call https://google.com to generate a simple external source in our traces. This was generated using https://start.spring.io on Spring Boot 3.0

I'll go through the differences now:

In Servlet, you must use logging.pattern.level as described in the docs to include the trace and span IDs in your logs. I am using %5p [${spring.application.name:},%X{traceId:-},%X{spanId:-}] as recommended from the docs. This creates a log like so:

However in Webflux, you are unable to get your trace and span IDs even with the recommended setting. You get a log line like so:

Notice how neither the trace nor the span are set in the log line. However it still does upload the span to the tracing backend.

The trace that is uploaded in Servlet looks how you would expect, a single trace with multiple spans.

In Webflux, multiple traces are submitted for the same request, when only separate spans should be created. This makes the correlation unusable as you can't tell which trace for the security filters is actually part of the request you are looking for.

(This is just 1 REST call)